To facilitate the quick detection of primary user and discover the available channel in a wireless cognitive network, a novel spectrum sensing mechanism was proposed. The mechanism conducted spectrum sensing simultaneously with data transmission to reduce the sense time, and exploited the interference cancellation technique to cancel out the self-interference. Furthermore, by combining the Fourier transform results of multiple signals with different sampling rates, it could quickly detect the behavior of the wideband channel. Finally, during the transmission, the primary user was detected in time by the well-known signature matching method. The experimental results show that, the sense time is reduced by 50%, and the throughput gain is as large as 100%. In summary, the proposed scheme can effectively reduce the overhead of spectrum sensing and improve the efficiency of cognitive communication.